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Definition 1. A problem of optimisation is one in which one tries to maximise or minimise a 
certain quantity called the objective, which depends on a finite number of variables. These may be 
either independent or related to one another through some constraints. 
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Definition 2. A mathematical programme is an optimisation problem in which the objective and 
the constraints are given as functions or mathematical relationship. In other words, 


optimise: z= f(x1,...,2n 


subject to: gi (a1,..-,2n) 


Some constraints are explicitly stated as requirements, others are hidden conditions. These latter 
need to be pin-pointed through the study and understanding of the model and its inputs. 
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Definition 3. A linear programme is a mathematical programme all the functions involved of 
which are linear. This means that, 


f(a1,---,@n) = %1 +--+ + Cnn 
Gi(T1,---, En) = A121 + +++ + Ginn 
where 7 = 1,...,m and c; and aij, j = 1,...,n, are constants. If there is an additional restriction 


on the input variables that they be all integers, then the optimisation problem is called an integer 
programme. A mathematical programme which is not a linear programme is said to be nonlinear. 
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Definition 4. A quadratic programme is a mathematical programme in which all the constraints 
are linear and the objective function is in quadratic form, which is in general, 


n n n 
f(x1, wed Dn) => Ss CigXi ty + ys djx; 
=k 


i=1 j=1 


where c,; and d; are constants. 
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Definition 5. A linear programme is said to be in standard form if all the constraints are 
equalities and if one feasible solution is known. In other words, our problem is now 
optimise: z= c?x 
subject to: Ax =b 
with: x>0 
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Definition 6. One may change any linear programme into the standard form by adding a slack 
variable to the left-hand side of a constraint of the form )7 a,j2; < 6; to obtain 


n 
) Ajj + Lp, = b; 
j=l 


where py > n and k = 1,2,.... Similarly one may add a surplus variable to the right-hand side of 
a constraint of the form )* a,;x2; > 6; to obtain \> ajax; = b; + tq, 


n 
DS dijtj — Bq = bi 
j=l 
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where q, > n and/=1,2,.... Next, all the slack and surplus variables are added to the objective 
function with zero coefficients. Then if we add an artificial variable to the left-hand side of each 
constraint where there is no slack variable, then the initial feasible solution is x» = b, where x 
is the vector of slack and artificial variables. The artificial variables are added to the objective 
function with a large negative coefficient —M. 


8 

Definition 7. A set of n vectors of m dimensions {v1,...,Vn} is said to be linearly dependent if 
there exist some constants a1,...,@,, not all of which are zero, such that 

Q1vi +--+: +anvyn = 0 (1) 

It is said to be linearly independent if the condition in Equation 1 implies ay = ag =-::-=a,y, = 0. 
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Theorem 1. Consider a set of n vectors of m dimensions. If n > m, then the set is linearly 
dependent. 


8 
Problem 1. Prove Theorem 1. 
8 
Definition 8. A vector v is called a convex combination of vectors v1,...,Vn if there exist some 
nonnegative constants 3,,...,3,, where 
Bit-:-+Bn=1 
such that 
v= 6ivit+-:-+Bnvn 
8 


Definition 9. A set of m-dimensional vectors is said to be convex if for any two vectors belonging 
to the set the line segment between them also belongs to the set. 
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Theorem 2. All points on the line segment joining any two vectors may be expressed as a convex 
combination of the two vectors. 


8 


Problem 2. Prove Theorem 2. 


8 


Definition 10. A vector v is called an eztreme point of a convex set if it can not be expressed 
as a convex combination of two other vectors in the set. 


In other words, an extreme point of a convex set K is a point x in K that cannot be written as 
x = 0y4+(1—8@)z with 0 <0< 1, y and z in K, and y ¥ 2z, that is to say, an extreme point is a 
point which is not an interior point of any line segment belonging to K. 

An equivalent definition of an extreme point is that x is an extreme point of a convex set K if 
K\ {x} is convex. 


8 


Definition 11. A metric space is a non-empty set X for which is defined a concept of distance. 
The distance d is called a metric on X, having such properties that, for any points x and y in X, 
we have d(x, y) > 0, and d(x, y) = 0 implies x = y; d(x, y) = d(y, x); and d(a,y) < d(x, z)+d(z, y). 


Let X be a metric space with metric d, let A be a subset of X and let x be any point of X. Then 
the distance from x to A is defined as 


d(x, A) = inf {d(x, a): a € A} 
whereas the diameter of A is defined as 
d(A) = sup {d(a1, a2) : a; andag € A} 
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Then a set is said to be bounded if its diameter is finite. 
Further, let 2p be a point in X and r a positive real number. Then the open sphere S,(ao) with 
centre xp and radius r is the subset of X defined by 


S-(a0) = {x : d(x, 29) < r} 


A point x in X is called a limit point of A if each open sphere centred on x contains at least one 
point of A different from x. A subset F' of X is said to be closed if it contains all its limit points. 
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Definition 12. A linear space, aka a vector space, is a non-empty set L on which is defined two 
binary processes, say addition and scalar multiplication. Addition is defined such that for any «, 
y and z in L, then «+ y is again in L; x+y =yt+a;a0+(yt+2z) = (x+y) + 2; there exists a 
unique identity element 0, aka zero element or the origin, such that x + 0 = x for every x; and 
there exists a unique inverse element —x for every x, such that «+(—ax) = 0. Scalar multiplication 
is defined with regard to scalars, some instances of which are real and complex numbers, such that 
for any scalar a and any x and y in L, az is again in L; a(a+y) =ar+ay; (a+ B)x = ar + Ba; 
(aB)x = a(Bx); and la = x, where 1 is the identity for scalar multiplication. A normed linear 
space is a linear space on which is defined a norm, that is a function which maps each element x in 
the space to a real number ||2|| in such a manner that ||z|| > 0, and ||a|| = 0 if and only if « = 0; 
lc + ll < llel| + llylls and |lee|] = al lla. 


8 
Theorem 3. A normed linear space is a metric space. 

8 
Problem 3. Prove Theorem 3. 

8 


Theorem 4. Any vector in a closed and bounded convex set with a finite number of extreme 
points can be expressed as a convex combinations of the extreme points. 


8 


Problem 4. Prove Theorem 4. 
8 


Definition 13. For two vectors, that is points, x and y in R”, we write x > y if and only if 
x; > y; for all 1 <i<n. A system of m weak linear inequalilties in n variables can be written as 
Ax > b, where A is an m x n matrix. A fundamental question concerning such system is whether 
it is consistent, that is to say, whether there exists some x such that Ax = b. A system may be 
inconsistent, or it may have a set of solutions which is unbounded. If we sketch our problem on a 
graph, we may see that it’s solution set is conve. 


8 


Theorem 5. The solution space of a set of simultaneous linear equations is a convex set the 
number of extreme points of which is finite. 


8 


Problem 5. Prove Theorem 5. 


8 


Theorem 6. Let S' be the set of all feasible solutions to the linear programme in standard form 
in Definition 5, in other words, S is the set of all vectors x that satisfy Ax = b and x > 0, where 
A is an m X n matrix. Then S is a convex set, and the number of its extreme points is finite. 
The objective function attains its optimum, provided that one exists, at an extreme point of S. If 
m <n, then the extreme points of S have at least n — m zero components. 


8 


Problem 6. Prove Theorem 6. 


Algorithm 1 Procedure for finding basic feasible solutions. 


Input: Ax = b, A is an m x n matrix, m <n, rank A =m 
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[ay -+- a,J<A 

(aja, +--+ +a¢n,an, = b) + (Ax = b) 

for i=m+1tondo 

endfor 

(@1,---;Un) < solve x1a; +--+ +a2,a, =b 


Definition 14. The simplex method is a matrix procedure which solves linear programmes of the 
standard form as described in Definition 5 where b > O. Starting from a basic feasible solution 
Xq we locate successively other basic feasible solutions giving better values for our objective. For 
minimisation programmes the method uses Table 1, for maximisation programmes the same table 
is also used but with the sign of entries in the bottom row reversed. 


8 
Table 1 Table used for minimisation programming in simplex method. 
ae 
ef 
Xo Co A b 
cP —co A —cb 
8 


Table 2 Description of the simplex method. 
while negative number exists in d do 


Locate the most negative number in the bottom row of the simplex table, excluding the last 
column. The column in which we find this number is called the work column. If more 
than one such column exist, choose one of them. 

Find the smallest of the ratios between the elements in the last column and the elements in 
the work column of the same row, if these latter are positive. The element in the work 
column that yields this smallest ratio is called the pivot element. If there are more than 
one of these, choose one. If none of the elements in the work column is positive, the 
programme has no solution. 

Using elementary row operations, convert the pivot element to 1 and reduce all other ele- 
ments in the work column to 0. 

Replace the z-variable in the pivot row and first column by the «-variable in the first row and 
pivot column. This new first column then becomes the current set of basic variables. 


endwhile 

The optimal solution is one in which all the basic variables assume the corresponding values in the 
last column, while the remaining variables are zero. The optimal value of the objective function is 
then the value of the last row and last column for a maximisation programme, and the negative of 
this value if the programme is one of minimisation. 


8 


Algorithm 2 gives a procedure for the simplex method. Here M represents a large positive 
integer, p a ratio, ca column, r a row, and xo contains all the basic variables. Also the last row in 
Table 1 is represented here by d = c? — cd A and e = —cdb. 


Algorithm 2 Algorithm for the simplex procedure. 
j <0 
while there exists a negative number in d do 
jegtl 
for 1=1 ton do 
{c’s} < (column number of the most negative number in the bottom row) 
(work column) «+ choose one of the {c}’s 
k © (work column) 
endfor 
Ppivot + M 
c+ 0 
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soln + 0 
for i= 1 tom do 
if (a;),, > 0 then 


soln «+1 


if p < Ppivot then 
ret 
endif 
endif 
endfor 
if soln = 0 then 
no solutions exist 

endif 

convert} A, such that (a;),,, = 1 and (a;);,, =0,1<i<m,i¥¢r 

(xo), <— Lk 
endwhile 
for i= 1 to m do 

(x0); sa (b;); 
endfor 
for i=m+1tondo 

x; +0 
endfor 
we ej 
if the programme is one of minimisation then 

Ze —2* 


endif 


Definition 15. The two-phase method is a procedure modified from the simplex method to cope 
with cases when artificial variables exist in the initial solution xo, in order to minimise the round-off 
errors that occur in the calculation. The last row in Table 1 in this case is d = ch-cp A = d,4+Mdbp, 
and consequently we have Table 3 which is used here. Algorithm 2 is then firstly applied to the 
last row, and then again to those elements directly above the zeros in that row. When an artificial 
variable is removed from the first column of the table, it ceases to be basic and may be removed 
from the top row of the table together with the entire column under it. When the last row contains 
only zeros, it may be deleted from the table. The programme has no solution if non-zero artificial 
variables are present in the final basic set. 


8 
Table 1 Table used for minimisation programming using the two-phase method. 
sot 
ee 
Xo Co A b 
d, —cib 
do 
8 
Definition 16. Given a linear programme in the variables 71,...,2,, there exists another linear 
programme associated with it, called its dual, which is in the variables w),...,wWm. The original 


programme is called the primal. The primal completely determines the form of its dual. The 
symmetric dual of a primal linear programme in the matrix form 


minimise: z=c!x 
subject to: Ax >b 


with: x>0 


+ With the use of elementary row operations. 
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is the linear programme 
maximise: z= b!w 
subject to: A’w<c 
with: w>0O 


The dual variables w1,...,Wm are known as shadow costs. The unsymmetric dual of the primal 


minimise: z=c x 
subject to: Ax =b 
with: x>0 
is 
maximise: z= b!w 
subject to: A’w<c 


The unsymmetric dual of the primal 


maximise: z=c x 
subject to: Ax=b 
with: x>0 
is 
minimise: z= b!’w 
subject to: A’w>c 


8 


Note 1. We may see from Definition 16 that the dual of a programme in standard form is not 
itself in standard form. These duals are said to be unsymmetric. 


8 


Theorem 7. If an optimal solution exists for either the primal or the dual programme, then the 
other programme also has an optimal solution. If the duality is symmetric, then the two functions 
have the same optimal value. If the duality if unsymmetric, then the optimal value of each function 
can be derived from that of the other. 
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